Identification of behavioral pattern of simulated transaction data

ABSTRACT

Embodiments can provide a method for identifying a behavioral pattern from simulated transaction data, including: simulating transaction data using a reinforcement learning model; identifying a behavioral pattern from the simulated transaction data; comparing the behavioral pattern with standard customer transaction data to determine whether the behavioral pattern is present in the standard customer transaction data. If the behavioral pattern is present in the standard customer transaction data, the behavioral pattern is applied in a model implemented on the cognitive system. The step of simulating transaction data further includes: providing standard customer transaction data representing a group of customers having similar transaction characteristics as a goal; and performing a plurality of iterations to simulate the standard customer transaction data, wherein the plurality of iterations is performed until a degree of similarity of simulated customer transaction data relative to the standard customer transaction data is higher than a first predefined threshold.

TECHNICAL FIELD

The present invention relates generally to identification of abehavioral pattern and more particularly, to identification of abehavioral pattern from simulated transaction data.

BACKGROUND

A financial crime detection system, e.g., IBM® Financial Crimes AlertsInsight with Watson™, can utilize cognitive analytics to help banks todetect money laundering and terrorist financing. The cognitive analyticsdifferentiate “normal” financial activities from “suspicious”activities, and use the differentiation information to build apredictive model for banks. A large set of real financial customer datais required to train the predictive model.

Since the real customer data is very sensitive, only a limited amount ofreal customer data can be provided by banks. However, in order to bestsimulate fraudulent situations and detect different types of financialcrimes, more simulated customer data, e.g., transaction data fortraining, which looks realistically, could produce a better predictivemodel.

SUMMARY

Embodiments provide a computer implemented method in a data processingsystem comprising a processor and a memory comprising instructions,which are executed by the processor to cause the processor to implementthe method for identifying a behavioral pattern from simulatedtransaction data on a cognitive system, the method includes: simulating,by the processor, transaction data using a reinforcement learning modelincluding an intelligent agent, a policy engine, and an environment;identifying, by the processor, a behavioral pattern from the simulatedtransaction data; comparing, by the processor, the behavioral patternwith standard customer transaction data to determine whether thebehavioral pattern is present in the standard customer transaction data,wherein the standard customer transaction data represents a group ofcustomers having similar transaction characteristics. If the behavioralpattern is present in the standard customer transaction data, the methodfurther includes applying, by the processor, the behavioral pattern in amodel implemented on the cognitive system. The step of simulatingtransaction data further includes: providing, by the processor, thestandard customer transaction data as a goal; performing, by theprocessor, a plurality of iterations to simulate the standard customertransaction data, wherein the plurality of iterations is performed untila degree of similarity of simulated customer transaction data relativeto the standard customer transaction data is higher than a firstpredefined threshold. In each iteration: conducting, by the intelligentagent, an action including a plurality of simulated transactions;comparing, by the environment, the action with the goal; providing bythe environment, a feedback associated with the action based on a degreeof similarity relative to the goal; and adjusting, by the policy engine,a policy based on the feedback.

Embodiments further provide a computer implemented method, wherein themodel implemented on the cognitive system is a fraudulent behaviordetection model, configured to determine whether the behavioral patternindicates a fraudulent behavior.

Embodiments further provide a computer implemented method, wherein themodel implemented on the cognitive system is the reinforcement learningmodel, configured to generate new simulated transaction data having thebehavioral pattern.

Embodiments further provide a computer implemented method, in eachiteration, further comprising: acquiring, by the processor, the standardcustomer transaction data from raw customer data through an unsupervisedclustering approach.

Embodiments further provide a computer implemented method, wherein eachsimulated transaction includes one or more of transaction type,transaction amount, transaction time, transaction location, transactionmedium, and a second party associated with the transaction.

Embodiments further provide a computer implemented method, the step ofidentifying a behavioral pattern further comprising identifying thebehavioral pattern based on a plurality of parameters includingbehavioral consistency, consistency volatility, and behaviorabnormality.

Embodiments further provide a computer implemented method, wherein thebehavioral pattern occurs during exploration in the reinforcementlearning model.

In another illustrative embodiment, a computer program productcomprising a computer usable or readable medium having a computerreadable program is provided. The computer readable program, whenexecuted on a processor, causes the processor to perform various onesof, and combinations of, the operations outlined above with regard tothe method illustrative embodiment.

In yet another illustrative embodiment, a system is provided. The systemmay comprise a training data harvesting processor configured to performvarious ones of, and combinations of, the operations outlined above withregard to the method illustrative embodiment.

Additional features and advantages of this disclosure will be madeapparent from the following detailed description of illustrativeembodiments that proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other aspects of the present invention are bestunderstood from the following detailed description when read inconnection with the accompanying drawings. For the purpose ofillustrating the invention, there is shown in the drawings embodimentsthat are presently preferred, it being understood, however, that theinvention is not limited to the specific instrumentalities disclosed.Included in the drawings are the following Figures:

FIG. 1 depicts a schematic diagram of one illustrative embodiment of acognitive system 100 implementing transaction data simulator, behavioralpattern identifier, and suspicious behavior detection model;

FIG. 2 depicts a schematic diagram of another illustrative embodiment ofa cognitive system 100 implementing transaction data simulator andbehavioral pattern identifier;

FIG. 3 depicts a schematic diagram of one illustrative embodiment of thetransaction data simulator 110;

FIG. 4 depicts a schematic diagram showing a plurality of simulatedtransactions from a simulated customer, according to embodiments herein;

FIG. 5 illustrates a flow chart of one illustrative embodiment of amethod 500 of simulating transaction data;

FIG. 6 illustrates a flow chart of one illustrative embodiment showing amethod 600 of identifying a fraudulent behavioral pattern;

FIG. 7 illustrates a flow chart of one illustrative embodiment showing amethod 700 of simulating a behavioral pattern; and

FIG. 8 is a block diagram of an example data processing system 800 inwhich aspects of the illustrative embodiments may be implemented.

DETAILED DESCRIPTION

As an overview, a cognitive system is a specialized computer system, orset of computer systems, configured with hardware and/or software logic(in combination with hardware logic upon which the software executes) toemulate human cognitive functions. These cognitive systems applyhuman-like characteristics to conveying and manipulating ideas which,when combined with the inherent strengths of digital computing, cansolve problems with high accuracy and resilience on a large scale. IBMWatson™ is an example of one such cognitive system which can processhuman readable language and identify inferences between text passageswith human-like accuracy at speeds far faster than human beings and on amuch larger scale. In general, such cognitive systems are able toperform the following functions:

-   -   Navigate the complexities of human language and understanding    -   Ingest and process vast amounts of structured and unstructured        data    -   Generate and evaluate hypotheses    -   Weigh and evaluate responses that are based only on relevant        evidence    -   Provide situation-specific advice, insights, and guidance    -   Improve knowledge and learn with each iteration and interaction        through machine learning processes    -   Enable decision making at the point of impact (contextual        guidance)    -   Scale in proportion to the task    -   Extend and magnify human expertise and cognition    -   Identify resonating, human-like attributes and traits from        natural language    -   Deduce various language specific or agnostic attributes from        natural language    -   High degree of relevant recollection from data points (images,        text, voice) (memorization and recall)    -   Predict and sense with situation awareness that mimics human        cognition based on experiences    -   Answer questions based on natural language and specific evidence

In one aspect, the cognitive system can be augmented with a transactiondata simulator, to simulate a set of customer transaction data from afinancial institution, e.g., a bank. The simulated customer transactiondata, even if it is not “actual” customer transaction data from thefinancial institution, can be used to train the predictive model foridentifying financial crimes.

The transaction data simulator combines a multi-layered unsupervisedclustering approach with interactive reinforcement learning (IRL) modelto create a large set of intelligent agents that have learned to behavelike “standard customers.”

In an embodiment, the multi-layered unsupervised clustering approachcreates a large set of standard customer transaction behaviors(extracted from real customer transaction data provided by a bank),using information including hundreds of attributes of “standardcustomers” over varying periods of time. Each standard customertransaction behavior can be associated with a group of customers havingsimilar transaction characteristics. An intelligent agent generates anartificial customer profile, and selects one of standard customertransaction behaviors to be combined with the generated artificialcustomer profile. In this way, the intelligent agent can simulate a“standard customer,” and learn to behave like the “standard customer.”The intelligent agent is then provided with a period of time (e.g., tenyears), during which the intelligent agent can observe an environment,e.g., past behaviors of the represented “standard customer”) and learnto perform “fake” customer transactions which are similar to standardcustomer transaction behavior of the represented “standard customer.”Each factor of the standard customer transaction behavior can bestatistic data. For example, the transaction amount of the standardcustomer transaction behavior can be a range of values, e.g., thetransaction amount of the standard customer transaction behavior is$20-$3,000. The transaction location of the standard customertransaction behavior can be provided statistically, e.g., 30% oftransaction locations are shopping malls, 50% of transaction locationsare restaurants, and 20% of transaction locations are gas stations. Thetransaction type of the standard customer transaction behavior can beprovided statistically, e.g., 20% of transaction types are checkpayment, 40% of transaction types are POS payment, 25% of transactiontypes are ATM withdrawal, and 15% of transaction types are wiretransfer. The transaction medium of the standard customer transactionbehavior can be provided statistically, e.g., 15% of transaction mediumsare cash, 45% of transaction mediums are credit card, 25% of transactionmediums are checking accounts, and 15% of transaction mediums arePayPal®.

In an embodiment, a large number of artificial customer profiles aregenerated from a plurality of real customer profile data. The realcustomer profile data can be provided by one or more banks. Each realcustomer profile can include an address of a customer; a name of acustomer (the customer can be a legal entity or individual); contactinformation such as a phone number, an email address, etc.; creditinformation, such as a credit score, a credit report, etc.; incomeinformation (e.g., an annual revenue of a legal entity, or a wage of anindividual), and the like. The real customer profile data are storedunder different categories. For example, commercial customers (i.e.,legal entities) can be divided into different categories based on thesize, product or service of the commercial customers. An artificialcustomer profile can be generated by randomly searching all the realcustomer profile data. For example, an artificial customer profile canbe generated by combining randomly selected information includingaddress, first name, second name, phone number, email address, creditscore, revenue or wage, etc. Thus, the generated artificial customerprofile extracts different pieces of information from real customerprofile data, and thus looks like a realistic customer profile.Financial transaction data is further simulated and associated with eachartificial customer profile.

In an embodiment, to protect privacy of real customers, compositeinformation, such as an address, a name, etc. can be split into aplurality of parts before the random selection. For example, the address“2471 George Wallace Street” can be parsed into 3 parts: [number]“2471,” [name] “George Wallace,” and [suffix] “Street.” These parts canbe randomly selected individually to form an artificial customerprofile. In a further embodiment, the composite information of anartificial customer profile, such as an address, a name, etc. iscompared to the composite information of a real customer profile. If thesimilarity degree is greater than a predefined threshold value, then theartificial customer profile is unacceptable and needs to be updateduntil the similarity degree is less than the predefined threshold value.

FIG. 1 depicts a schematic diagram of one illustrative embodiment of acognitive system 100 implementing transaction data simulator 110,behavioral pattern identifier 112, and suspicious behavior detectionmodel 116. The cognitive system 100 is implemented on one or morecomputing devices 104 (comprising one or more processors and one or morememories, and potentially any other computing device elements generallyknown in the art including buses, storage devices, communicationinterfaces, and the like) connected to the computer network 102. Thecomputer network 102 includes multiple computing devices 104 incommunication with each other and with other devices or components viaone or more wired and/or wireless data communication links, where eachcommunication link comprises one or more of wires, routers, switches,transmitters, receivers, or the like. Other embodiments of the cognitivesystem 100 may be used with components, systems, sub-systems, and/ordevices other than those that are depicted herein. The computer network102 includes local network connections and remote connections in variousembodiments, such that the cognitive system 100 may operate inenvironments of any size, including local and global, e.g., theInternet. The cognitive system 100 is configured to implementtransaction data simulator 110 that can simulate standard customertransaction data 106 (i.e., a standard customer transaction behavior).The transaction data simulator 110 can generate a large set of simulatedcustomer transaction data 108 based on the standard customer transactiondata 106, so that the simulated customer transaction data 108 looks likereal customer transaction data. In an embodiment, the standard customertransaction data 106 is obtained through unsupervised clusteringapproach. Raw customer data including a large amount of customertransaction data is provided by one or more banks, and a large set ofsmall groups representing different characteristics of bank customersare clustered or grouped from the raw customer data through unsupervisedclustering approach. Each small group includes transaction data fromcustomers having similar characteristics. For example, group Arepresents customers who are single attorneys practicing patent law inNew York, while group B represents customers who are married attorneyspracticing commercial law in New York.

In an embodiment, a behavioral pattern identifier 112 is alsoimplemented on the cognitive system 100. The behavioral patternidentifier 112 can identify one or more behavioral patterns 114 from thesimulated customer transaction data 108. For example, in the simulatedcustomer transaction data 108 within a period of one year, thebehavioral pattern identifier 112 may find that at the end of eachmonth, a transaction having a transaction amount of $1500 was made forrent. For another embodiment, the behavioral pattern identifier 112 mayidentify a save-and-spend (spending quickly upon arrival of salary)behavioral pattern from the simulated customer transaction data 108. Inan embodiment, the identified behavioral pattern 114 can be sent to afraudulent behavior detection model 116 for further analysis. Forexample, the fraudulent behavior detection model 116 may determinewhether the behavioral pattern 114 indicates a fraudulent behavior.

FIG. 2 depicts a schematic diagram of another illustrative embodiment ofa cognitive system 100 implementing transaction data simulator 110 andbehavioral pattern identifier 112. In this embodiment, the identifiedbehavioral pattern 114 is sent back to the transaction data simulator110, so that the transaction data simulator 110 can generate newsimulated customer transaction data 108 having the identified behavioralpattern 114.

FIG. 3 depicts a schematic diagram of one illustrative embodiment of thetransaction data simulator 110. The transaction data simulator 110utilizes reinforcement learning techniques to simulate financialtransaction data. The transaction data simulator 110 includesintelligent agent 202, and environment 204. The intelligent agent 202randomly selects a standard transaction behavior 220 (i.e. goal 220)representing a group of “customers” having similar transactioncharacteristics, and associates the standard transaction behavior with arandomly selected artificial customer profile 218. The intelligent agent202 takes an action 212 in each iteration. In this embodiment, theaction 212 taken in each iteration includes conducting a plurality oftransactions in a single day. Each transaction has the informationincluding transaction type (e.g., Automated Clearing House (ACH)transfer, check payment, Wire transfer, Automated Teller Machine (ATM)withdrawal, Point of Sale (POS) payment, etc.); transaction amount;transaction time; transaction location; transaction medium (e.g., cash,credit card, debit card, PayPal®, checking account, etc.); the secondparty who is related to the transaction (e.g., a person who receives thewire transferred payment), and the like. The environment 204 takes theaction 212 as input, and returns reward 214 (or feedback) and state 216from environment 204 as the output. The reward 214 is the feedback bywhich the success or failure of the action 212 is measured. In thisembodiment, the environment 204 compares the action 212 with goal 220(e.g., standard transaction behavior). If the action 212 deviates fromthe goal 220 beyond a predefined threshold, then the intelligent agent202 is penalized, while if the action 212 deviates from the goal 220within a predefined threshold (i.e., the action 212 is similar to thegoal 220), the intelligent agent 202 is rewarded. The action 212 iseffectively evaluated, so that the intelligent agent 202 can improve thenext action 212 based on the reward 214. In this embodiment, theenvironment 204 is a set of all old actions taken by the intelligentagent 202, i.e., the environment 204 is a set of all old simulatedtransactions. The intelligent agent 202 observes the environment 204,and gets information about the old transactions, e.g., the number oftransactions that have been made within a day, a week, a month, or ayear; each transaction amount, account balance, each transaction type,and the like. The policy engine 206 can adjust the policy based on theobservations, so that the intelligent agent 202 can take a better action212 in the next iteration.

The intelligent agent 202 further includes policy engine 206, configuredto adjust a policy based on the state 216 and the reward 214. The policyis a strategy that the intelligent agent 202 employs to determine thenext action 212 based on the state 216 and the reward 214. The policy isadjusted, aiming to get a higher reward 214 for the next action 212taken by the intelligent agent 202. The policy includes a set ofdifferent policy probabilities or decision-making probabilities whichcan be used to decide whether a transaction is going to be performed ina particular day or not, the number of transactions per day, transactionamount, transaction type, transaction party, etc. In reinforcementlearning model, outcome of events are random, and a random numbergenerator (RNG) is a system that generates random numbers from a truesource of randomness. In an example, the maximum number of transactionsper day is 100, and the maximum transaction amount is $15 million. Inthe first iteration, a random transaction with transaction amount of $15million to Zimbabwe is made by the intelligent agent 202. This action212 deviates far from the goal 220 (e.g., transaction made by marriedattorneys practicing commercial law in Maine), and thus this action 212is penalized (i.e., the reward 214 is negative). The policy engine 206is trained to adjust the policy, so that a different transaction whichis closer to the goal 220 can be made. With more iterations,transactions which are similar to the goal 220 can be simulated by the“smarter” policy engine 206. As shown in FIG. 4, a plurality oftransactions from the customer “James Culley” are simulated, and thesimulated transaction data is similar to the goal 220.

As shown in FIG. 3, in an embodiment, one feedback loop (i.e., oneiteration) corresponds to one “day” of actions (i.e., one “day” ofsimulated transactions). During a period of time, e.g., ten years, theintelligent agent 202 learns how to take an action 212 to get a reward214 as high as possible. The number of iterations corresponds to theduration of time. For example, ten years correspond to 10×365=3650iterations. Reinforcement learning model judges the actions 212 by theresults that the actions 212 produce. It is goal 220 oriented, and itsaim is to learn sequences of actions 212 that will lead the intelligentagent 202 to achieve its goal 220, or maximize its objective function.

In an embodiment, the transaction data simulator 110 further includesupdater 210. A new action 212 is performed in each iteration. Theupdater 210 updates the environment 204 with the action 212 taken by theintelligent agent 202 after each iteration. The action 212 taken in eachiteration is added into the environment 204 by the updater 210. In anembodiment, the transaction data simulator 110 further includes pruner208, configured to prune the environment 204. In an embodiment, thepruner 208 can remove one or more undesired actions. For example,actions 212 which are taken in the first ten iterations are removed,because these ten iterations deviate far from the goal 220, and thedegree of similarity is below a predefined threshold. In anotherembodiment, a full re-initialization of the transaction data simulator110 can be performed to remove all the accumulated actions in theenvironment 204, so that the intelligent agent 202 can start over again.

FIG. 5 illustrates a flow chart of one illustrative embodiment showing amethod 500 of simulating transaction data. At step 502, standardcustomer transaction behavior data is provided as goal 220. The standardcustomer transaction behavior represents a group of customers havingsimilar transaction characteristics. The standard customer transactionbehavior is obtained through unsupervised clustering approach.

At step 504, an action 212 is taken to conduct a plurality oftransactions in an iteration representing e.g., a single day (e.g., 100transactions per day). Each transaction has the information includingtransaction type, transaction amount, transaction time, transactionlocation, transaction medium, the second party who is associated withthe transaction, and the like.

At step 506, the environment 204 compares the goal 220 with the action212 taken in this iteration, rewards or penalizes the action 212 basedon similarity to or deviation from the goal 220. The threshold or ruleto decide whether the action 212 is similar to the goal 220 or not, ispredefined, and can be adjusted based on how similar to the goal 220 theuser prefers.

At step 508, the environment 204 is updated to include the action 212 inthe present iteration. The environment 204 includes a set of all oldactions.

At step 510, the policy engine 206 adjusts a policy for determining thenext action 212 based on the reward 214 (i.e., reward or penalty). Thepolicy is made based on a variety of factors, e.g., probability ofoccurrence of a transaction, the number of transactions per day,transaction amount, transaction type, transaction party, transactionfrequency of each transaction type, an upper bound and a lower bound foreach transaction, transaction medium, and the like. The policy canadjust weights of these factors based on the reward 214 in eachiteration.

At step 512, in a new iteration, the intelligent agent 202 takes a newaction 212. The steps 504 to 512 are repeated until the action 212 issimilar enough to the goal 220 (step 514). For example, the transactionamount specified in the goal 220 is $20-$3000. If the transaction amountof each transaction in the action 212 falls within the range of$20-$3000, then the action 212 is similar enough to the goal 220.

Since the standard customer transaction data 106 may include abnormaldata, e.g., a fraudulent transaction, the simulated customer transactiondata 108 may also include abnormal data, because the simulated customertransaction data 108 is similar to the standard customer transactiondata 106. In reinforcement learning model, the intelligent agent 202explores the environment 204 randomly or stochastically, learns a policyfrom its experiences, and updates the policy as it explores to improvethe behavior (i.e., transaction) of the intelligent agent 202. In anembodiment, a behavioral pattern (e.g., spending “splurges” untilrunning out of savings, or experiencing “buyer's remorse” on one bigpurchase, etc.), as opposed to random actions, may emerge during RNGbased exploration. An abnormal behavioral pattern may indicate afraudulent transaction. For example, a simulated customer James Culleymay generally make transactions having a transaction amount below$1,000. Suddenly, there is a transaction having a transaction amount of$5,000, and this suspicious transaction may be a fraudulent transaction(e.g., the credit card of James Culley is stolen, or the checkingaccount of James Culley is hacked).

There is a behavioral pattern that naturally emerges or occurs duringexploration. For example, as shown in FIG. 4, the simulated customerJames Culley received an amount of $12,387.71 in a checking account onJan. 1, 2014. James Culley spent $474.98 on Jan. 3, 2014, $4,400 onJanuary 3, and $3,856.55 on Jan. 4, 2014 through a debit card associatedwith the checking account. In the next Month, James Culley received anamount of $12,387.71 in the checking account on Feb. 1, 2014. JamesCulley spent $4,500 on Feb. 2, 2014, and $1,713.91 on February 3 throughthe debit card associated with the checking account, and transferred$8,100 out of the checking account on Jun. 27, 2014. In this example,this simulated customer James Culley has a tendency of save-and-spend,and occasionally has a big purchase. The behavioral pattern makes thissimulated customer James Culley behave more realistically (i.e., lookmore like a real customer, rather than a robot). A plurality ofparameters, such as “behavioral consistency” (the degree of behavioralconsistency in a period of time), “consistency volatility” (frequency ofbehavior change), “behavior abnormality” (deviation from regulartransaction behaviors), etc. are generated by the policy engine 206, andused to show a different personality or behavioral pattern of eachsimulated customer.

FIG. 6 illustrates a flow chart of one illustrative embodiment showing amethod 600 of identifying a fraudulent behavioral pattern. Referring toFIG. 1 and FIG. 6, at step 602, the simulated customer transaction data108 generated in the method 500 of FIG. 5 is provided to the behavioralpattern identifier 112. At step 604, the behavioral pattern identifier112 identifies a behavioral pattern 114 from the simulated customertransaction data 108. At step 606, a user or an algorithm can comparethe behavioral pattern 114 with the standard customer transaction data106, and check whether the behavioral pattern 114 is present in thestandard customer transaction data 106. At step 608, if the behavioralpattern 114 is also present in the standard customer transaction data106, then it means that the behavioral pattern 114 is simulated andidentified correctly, and thus the behavioral pattern 114 can beutilized in different scenarios. In an embodiment, the behavioralpattern 114 can be provided to a fraudulent behavior detection model 116to determine whether the behavioral pattern 114 indicates a fraudulentbehavior. If the behavioral pattern 114 is not present in the standardcustomer transaction data 106, then the behavioral pattern 114 is aresult of random transactions during exploration in the reinforcementlearning model.

FIG. 7 illustrates a flow chart of one illustrative embodiment showing amethod 700 of simulating a behavioral pattern. Referring to FIG. 1 andFIG. 7, the step 702 to the step 706 are the same as the step 602 to thestep 606. At step 708, the behavioral pattern 114 can be input back intothe transaction data simulator 110. The transaction data simulator 110can generate simulated customer transaction data 108 having thebehavioral pattern 114.

The transaction data simulator 110 uses abstracted or aggregated realcustomer data to simulate customer data that is representative of realcustomers. The transaction data simulator 110 can provide a large set ofsimulated customer data (i.e., simulated transaction data in combinationwith an artificial customer profile) that can be used to train apredictive model for detecting abnormal customer behaviors. Further, thesimulated customer data is generated based on abstracted data of thereal raw customer data, rather than the real raw customer data itself,and thus it is impossible to derive actual transaction actions of anyreal customer. Additionally, the transaction data simulator 110 allowsgeneration of a behavioral pattern for each simulated customer duringiterations.

FIG. 8 is a block diagram of an example data processing system 800 inwhich aspects of the illustrative embodiments are implemented. Dataprocessing system 800 is an example of a computer, such as a server orclient, in which computer usable code or instructions implementing theprocess for illustrative embodiments of the present invention arelocated. In one embodiment, FIG. 8 represents a server computing device,such as a server, which implements the cognitive system 100 describedherein.

In the depicted example, data processing system 800 can employ a hubarchitecture including a north bridge and memory controller hub (NB/MCH)801 and south bridge and input/output (I/O) controller hub (SB/ICH) 802.Processing unit 803, main memory 804, and graphics processor 805 can beconnected to the NB/MCH 801. Graphics processor 805 can be connected tothe NB/MCH 801 through, for example, an accelerated graphics port (AGP).

In the depicted example, a network adapter 806 connects to the SB/ICH802. An audio adapter 807, keyboard and mouse adapter 808, modem 809,read only memory (ROM) 810, hard disk drive (HDD) 811, optical drive(e.g., CD or DVD) 812, universal serial bus (USB) ports and othercommunication ports 813, and PCI/PCIe devices 814 may connect to theSB/ICH 802 through bus system 816. PCI/PCIe devices 814 may includeEthernet adapters, add-in cards, and PC cards for notebook computers.ROM 810 may be, for example, a flash basic input/output system (BIOS).The HDD 811 and optical drive 812 can use an integrated driveelectronics (IDE) or serial advanced technology attachment (SATA)interface. A super I/O (SIO) device 815 can be connected to the SB/ICH802.

An operating system can run on processing unit 803. The operating systemcan coordinate and provide control of various components within the dataprocessing system 800. As a client, the operating system can be acommercially available operating system. An object-oriented programmingsystem, such as the Java′ programming system, may run in conjunctionwith the operating system and provide calls to the operating system fromthe object-oriented programs or applications executing on the dataprocessing system 800. As a server, the data processing system 800 canbe an IBM® eServer™ System P® running the Advanced Interactive Executiveoperating system or the LINUX® operating system. The data processingsystem 800 can be a symmetric multiprocessor (SMP) system that caninclude a plurality of processors in the processing unit 803.Alternatively, a single processor system may be employed.

Instructions for the operating system, the object-oriented programmingsystem, and applications or programs are located on storage devices,such as the HDD 811, and are loaded into the main memory 804 forexecution by the processing unit 803. The processes for embodiments ofthe cognitive system 100, described herein, can be performed by theprocessing unit 803 using computer usable program code, which can belocated in a memory such as, for example, main memory 804, ROM 810, orin one or more peripheral devices.

A bus system 816 can be comprised of one or more busses. The bus system816 can be implemented using any type of communication fabric orarchitecture that can provide for a transfer of data between differentcomponents or devices attached to the fabric or architecture. Acommunication unit such as the modem 809 or the network adapter 806 caninclude one or more devices that can be used to transmit and receivedata.

Those of ordinary skill in the art will appreciate that the hardwaredepicted in FIG. 8 may vary depending on the implementation. Otherinternal hardware or peripheral devices, such as flash memory,equivalent non-volatile memory, or optical disk drives may be used inaddition to or in place of the hardware depicted. Moreover, the dataprocessing system 800 can take the form of any of a number of differentdata processing systems, including but not limited to, client computingdevices, server computing devices, tablet computers, laptop computers,telephone or other communication devices, personal digital assistants,and the like. Essentially, data processing system 800 can be any knownor later developed data processing system without architecturallimitation.

The system and processes of the figures are not exclusive. Othersystems, processes, and menus may be derived in accordance with theprinciples of embodiments described herein to accomplish the sameobjectives. It is to be understood that the embodiments and variationsshown and described herein are for illustration purposes only.Modifications to the current design may be implemented by those skilledin the art, without departing from the scope of the embodiments. Asdescribed herein, the various systems, subsystems, agents, managers, andprocesses can be implemented using hardware components, softwarecomponents, and/or combinations thereof. No claim element herein is tobe construed under the provisions of 35 U.S.C. 112 (f), unless theelement is expressly recited using the phrase “means for.”

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a head disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network(LAN), a wide area network (WAN), and/or a wireless network. The networkmay comprise copper transmission cables, optical transmission fibers,wireless transmission, routers, firewalls, switches, gateway computers,and/or edge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including anobject-oriented programming language such as Java, Smalltalk, C++ or thelike, and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computer,or entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including LAN or WAN, or the connection may be made toan external computer (for example, through the Internet using anInternet Service Provider). In some embodiments, electronic circuitryincluding, for example, programmable logic circuitry, field-programmablegate arrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatuses(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operations steps to be performed on the computer,other programmable apparatus, or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical functions. In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The present description and claims may make use of the terms “a,” “atleast one of,” and “one or more of,” with regard to particular featuresand elements of the illustrative embodiments. It should be appreciatedthat these terms and phrases are intended to state that there is atleast one of the particular feature or element present in the particularillustrative embodiment, but that more than one can also be present.That is, these terms/phrases are not intended to limit the descriptionor claims to a single feature/element being present or require that aplurality of such features/elements be present. To the contrary, theseterms/phrases only require at least a single feature/element with thepossibility of a plurality of such features/elements being within thescope of the description and claims.

In addition, it should be appreciated that the following descriptionuses a plurality of various examples for various elements of theillustrative embodiments to further illustrate example implementationsof the illustrative embodiments and to aid in the understanding of themechanisms of the illustrative embodiments. These examples are intendedto be non-limiting and are not exhaustive of the various possibilitiesfor implementing the mechanisms of the illustrative embodiments. It willbe apparent to those of ordinary skill in the art in view of the presentdescription that there are many other alternative implementations forthese various elements that may be utilized in addition to, or inreplacement of, the example provided herein without departing from thespirit and scope of the present invention.

Although the invention has been described with reference to exemplaryembodiments, it is not limited thereto. Those skilled in the art willappreciate that numerous changes and modifications may be made to thepreferred embodiments of the invention and that such changes andmodifications may be made without departing from the true spirit of theinvention. It is therefore intended that the appended claims beconstrued to cover all such equivalent variations as fall within thetrue spirit and scope of the invention.

We claim:
 1. A computer implemented method in a data processing systemcomprising a processor and a memory comprising instructions, which areexecuted by the processor to cause the processor to implement the methodfor identifying a behavioral pattern from simulated transaction data ona cognitive system, the method comprising: simulating, by the processor,transaction data using a reinforcement learning model including anintelligent agent, a policy engine, and an environment; identifying, bythe processor, a behavioral pattern from the simulated transaction data;comparing, by the processor, the behavioral pattern with standardcustomer transaction data to determine whether the behavioral pattern ispresent in the standard customer transaction data, wherein the standardcustomer transaction data represents a group of customers having similartransaction characteristics, and if the behavioral pattern is present inthe standard customer transaction data, applying, by the processor, thebehavioral pattern in a model implemented on the cognitive system,wherein the step of simulating transaction data further comprises:providing, by the processor, the standard customer transaction data as agoal; performing, by the processor, a plurality of iterations tosimulate the standard customer transaction data, wherein the pluralityof iterations is performed until a degree of similarity of simulatedcustomer transaction data relative to the standard customer transactiondata is higher than a first predefined threshold; in each iteration:conducting, by the intelligent agent, an action including a plurality ofsimulated transactions; comparing, by the environment, the action withthe goal; providing by the environment, a feedback associated with theaction based on a degree of similarity relative to the goal; andadjusting, by the policy engine, a policy based on the feedback.
 2. Themethod as recited in claim 1, wherein the model implemented on thecognitive system is a fraudulent behavior detection model, configured todetermine whether the behavioral pattern indicates a fraudulentbehavior.
 3. The method as recited in claim 1, wherein the modelimplemented on the cognitive system is the reinforcement learning model,configured to generate new simulated transaction data having thebehavioral pattern.
 4. The method as recited in claim 1, in eachiteration, further comprising: acquiring, by the processor, the standardcustomer transaction data from raw customer data through an unsupervisedclustering approach.
 5. The method as recited in claim 1, wherein eachsimulated transaction includes one or more of transaction type,transaction amount, transaction time, transaction location, transactionmedium, and a second party associated with the transaction.
 6. Themethod as recited in claim 1, the step of identifying a behavioralpattern further comprising identifying the behavioral pattern based on aplurality of parameters including behavioral consistency, consistencyvolatility, and behavior abnormality.
 7. The method as recited in claim1, wherein the behavioral pattern occurs during exploration in thereinforcement learning model.
 8. A computer program product foridentifying a behavioral pattern from simulated transaction datagenerated on a cognitive system, the computer program product comprisinga computer readable storage medium having program instructions embodiedtherewith, the program instructions executable by a processor to causethe processor to: simulate transaction data using a reinforcementlearning model including an intelligent agent, a policy engine, and anenvironment; identify a behavioral pattern from the simulatedtransaction data; compare the behavioral pattern with standard customertransaction data to determine whether the behavioral pattern is presentin the standard customer transaction data, wherein the standard customertransaction data represents a group of customers having similartransaction characteristics, and if the behavioral pattern is present inthe standard customer transaction data, apply the behavioral pattern ina model implemented on the cognitive system, wherein the step ofsimulating transaction data further comprises: provide the standardcustomer transaction data as a goal; and perform a plurality ofiterations to simulate the standard customer transaction data, whereinthe plurality of iterations is performed until a degree of similarity ofsimulated customer transaction data relative to the standard customertransaction data is higher than a first predefined threshold; in eachiteration: conduct, by the intelligent agent, an action including aplurality of simulated transactions; compare, by the environment, theaction with the goal; provide, by the environment, a feedback associatedwith the action based on a degree of similarity relative to the goal;and adjust, by the policy engine, a policy based on the feedback.
 9. Thecomputer program product of claim 8, wherein the model implemented onthe cognitive system is a fraudulent behavior detection model,configured to determine whether the behavioral pattern indicates afraudulent behavior.
 10. The computer program product of claim 8,wherein the model implemented on the cognitive system is thereinforcement learning model, configured to generate new simulatedtransaction data having the behavioral pattern.
 11. The computer programproduct of claim 8, wherein the program instructions executable by theprocessor further cause the processor to: acquire the standard customertransaction data from raw customer data through an unsupervisedclustering approach.
 12. The computer program product of claim 8,wherein each simulated transaction includes one or more of transactiontype, transaction amount, transaction time, transaction location,transaction medium, and a second party associated with the transaction.13. The computer program product of claim 8, wherein the step ofidentifying a behavioral pattern further comprising identifying thebehavioral pattern based on a plurality of parameters includingbehavioral consistency, consistency volatility, and behaviorabnormality.
 14. The computer program product of claim 8, wherein thebehavioral pattern occurs during exploration in the reinforcementlearning model.
 15. A system for identifying a behavioral pattern fromsimulated transaction data generated on a cognitive system, the systemcomprising: a processor configured to: simulate transaction data using areinforcement learning model including an intelligent agent, a policyengine, and an environment; identify a behavioral pattern from thesimulated transaction data; compare the behavioral pattern with standardcustomer transaction data to determine whether the behavioral pattern ispresent in the standard customer transaction data, wherein the standardcustomer transaction data represents a group of customers having similartransaction characteristics, and if the behavioral pattern is present inthe standard customer transaction data, apply the behavioral pattern ina model implemented on the cognitive system, wherein the step ofsimulating transaction data further comprises: provide the standardcustomer transaction data as a goal; and perform a plurality ofiterations to simulate the standard customer transaction data, whereinthe plurality of iterations is performed until a degree of similarity ofsimulated customer transaction data relative to the standard customertransaction data is higher than a first predefined threshold; in eachiteration: conduct, by the intelligent agent, an action including aplurality of simulated transactions; compare, by the environment, theaction with the goal; provide, by the environment, a feedback associatedwith the action based on a degree of similarity relative to the goal;and adjust, by the policy engine, a policy based on the feedback. 16.The system of claim 15, wherein the model implemented on the cognitivesystem is a fraudulent behavior detection model, configured to determinewhether the behavioral pattern indicates a fraudulent behavior.
 17. Thesystem of claim 15, wherein the model implemented on the cognitivesystem is the reinforcement learning model, configured to generate newsimulated transaction data having the behavioral pattern.
 18. The systemof claim 15, wherein the program instructions executable by theprocessor further cause the processor to: acquire the standard customertransaction data from raw customer data through an unsupervisedclustering approach.
 19. The system of claim 15, wherein each simulatedtransaction includes one or more of transaction type, transactionamount, transaction time, transaction location, transaction medium, anda second party associated with the transaction.
 20. The system of claim15, the step of identifying a behavioral pattern further comprisingidentifying the behavioral pattern based on a plurality of parametersincluding behavioral consistency, consistency volatility, and behaviorabnormality.